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ABSTRACT _ 

Recent developnents in faculty organization and 
accountability are discussed separately in terms of their T 
implications for eva luation of academic programs and departments . 
Faculty organization if examined in relation to each components* 

(individual faculty members, departments, colleges, and other 
assorted units) acce ssibility to and utilization of feedback data 
assessing their performance and effectiveness. Recommendations for 
checking the possible inhibiting effects of departmental autonomy 
upon department program evaluation are advanced. Suggestions for a 
nation-wide effort (coordinated by national organizations) to 
establish goal consensus and accountability criteria among 
institutions according to field, are made. Through organizations such 
as the Speech Communication Association and Association of 
Departments and Administrators in Speech Communication, the general 
public- and decision-makers in institutions of higher learning can be 
educated about the value of increasing the communication skills and 
sophistication of individuals, groups, and organizations, 

(Author/LG) . 
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Every parent who has ^ ever managed to live through the teen- 

his ' - : 

age years of ibfe^sar children is more than f amiliar with^ the evane- 
scent norms that shape their children one way one week/ and 
another way the next. We; joke and laugh about these teenr age fads — 
whenever we can -^do so through our tears, that is, ^ . 

Such fads that sweep; the teen-age coiranunity are cause, enough 
^tp weep^v Ev^eh tlwughv\ etre^ the liadSL 

^hcri: alSbst as^ 

Vijijreqent tajnes^ ai^^ . 
^hen progr^ 

"jacqoAint^ili /in the 

^^publSLc sch^ /v-^^ v:^:x^^^ 

V Thou I have observe^ 

yf|ar;sv: I haye^ npt y^ ydi forces 
:^at give rjse -tp>^^them^ 

declinev Witfc eyd\icationat f ^ ; 
I can observ^ sprtUB p^^^ srtr^ ^. 

rise of many of -th^ the push f or/ef f iciency^/. f or e.ducating > 

more students or educat in|f - th^ better - with less money /-As eacK 
f ad approaches its: pe^/ we can^usudlly^^^^s^^ of studies 

purporting to di^naonstrat^^ the usefulness of the^^ particular practice. 
-TM^ most ofj the studies :ha^^ 

?not been weil done ;and that /- "ther^f are ques- _ 
tionable. weill-dbneV. ^1^^ and thC results 
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of these usually mark the beginning of the decline. Such a large- 
scale, study may well have just been dene on perjform.ancG contracting. 
After a great many small demonstration stadies/ the United States 
Office of Economic Opportunity was so confident of the success of 
performance contracting that it spent over six--million dollars in 
a large/ nation-wide study of per foinnnance contracting with under- 
achieving students. Thirty-one companies bid for the opportunity 
to participate/ even though they would be paid only for each 
student who was able to gain one school year of growth in reading 
and arithmetic during the €«^school- year . Half or more of those 
:compcmies ^that, particip^^^ out pf business . The pupils 

in the experiment fell far short of the goal. Even more important/ • 
when; the _gains on the^^ cbmpared t b the gairig ;of 

an equal :n\amber pt: c no effects 

^X>f pe^f orm^ 

1 am npt^^suggiestom 
or -that^ we ignore: them.^^^^ them the^^^v 

seeds for new blossoms in the educational gar den-^new growth that C Ah 
contribute to the achieyement of prur; yarie^^^^^ must 
:aypid;^pthv;Jtind^::of ^ M of the 

academic^ cpiptunity^-cp^ i^W^ dey^^ Sppe 

of us tend h^itually ,^p poph- 

that it is not/ in fact/ new at all or else that it is useless or 
even destructive. Others of us tend/ just as habituaj.ly/ to immedi- 
ately gather every new development to our- institutional bosoms / 
either, because we perceive: that goyernment and foundation . grants 
tend to "go tpi. innovator s^^or else because j^^e^ar^^^ ly di^atis- 

fied. with what we are doing- cmdV :s6: -ar^- lalways. ready ta try something 
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new. 1 hope 'that we here can find some middle ground — that we dan . 
analyze these new developments to discover the potential strengths 
and weaknesses and to develop means of capitalizing on the fonner 
while minimizing the latter. This is the framework within which_ 
I will be discussing faculty organization/ accovintability/ and 
the evaluation of academic programs.: 

The recent developments in faculty organization .and in account- 
ability which we have been discussing at this conference have 
qu£ter different sorts of implications for the evaluation of aca-_ ; 
demic programs or of departments. Therefore , I will consider them ^ 
separately^ f ir^t faculty orx[anizati<^ : 



■ :^^V^> ; FACULTY ORGJ^NiZATION . / 

, We. liave; tall^ 

.of faculty organization, the union or something which approaches 
the union. I believe that we should also consider another type 
of , f acul^^^^^ that withi;^ ihstitUtibn-- the prgan- 

ization into departments / ' colleges/ and other assorted units 
Both of these :ki^i5^ o£ brg^aMza^ can: -aff ect the^ degrees of Vfrfeje- 
^dom_ that we have for an i^d of^ academic progra^^ 

and departments. . i " . 

As teaching unions gather strength/ they may inhibit or • attempt 
to inhibit the gathering of some of the kinds of data that are 
needed for an adequate assessment of a department • s af fectiveness. 
vSince that effectiveness i^^^^ 

tiveness of the faculty members within the department/ we must 
have the freedom to asses s as cpmpletely as we can the effective- 
ness of each iridividtial f aculty member. > We : mu^t be:_ able to_ get i 
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or 



Student ratings of each individual teacher^ as well as student 
ratings of courses and programs. Vie must also attempt to get some 
comparative data on the performance of students who have worked 
under each faculty, member • (I will return to this point l«ter 
when I discuss accountability.) Any restrictions which union 
contracts place upon the. gathering and use. of such data will limit 
the validity of departoental evaluat ions • 

Departmental and college structures may inhibit such evalua-^ 
trons in pother .way. Though I am a fervent .believer in aepart- 
mental autonomy, as I assume you are / we must recognize that such 
r autonomy may.: e lirainate wh^ to^ be some^ of : the ma j or criteria . 

±4 the evaluation of a^^rd^ such: criterion is the deg^^^^ 

vto ^which: ^ - Separ^^^ q^irings^jser^ 

^other departmei^^^^ or — 

i)ri5iar i iy its ma j or s..- The o ther rel £s _ how wel 1-a 

department uses /what li^^^^^ We^ 

must get^ a^^y; frcttii^ theiscS^ ieacls 
us to berieye that only the^fajsulty in a deparly:^ 
to deteanriine what a "^depart^ ought , to teacK or who we ought: to 
teachr it to. Our. departi^ and/ the^re 

ought to i>e:;e Vt^uati^ not; pnl^orv . thei . teasis^ of: the^ jeyidence o^ 
how>^ff ectively: we yare meeting, ihej lea^^ majpirsy 
but alsofwhetHer we ^ a^^^ meeting the: learn^^ needs, of ■students in 
American Civirization (with our Public Address coursesl"/ of students 

^in pre-iaw (with our Argument at ion and Persuasion courses),/ of 
students in English (with our dramatic literatu^ jand Rhetprip^ 
Theory courses) / ^M^^ Journali^; and Art (i^ith' bur 

broadcasting, and f ilm /cpursels)./ etc.. W^^^ be evaluated 
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on how well we use the courses from other departments to educate 
our students more effectively and efficiently. -We should' be asked 
whether we insist on teaching our own statistics courses when 
there are better statistics courses taught by" the department of 
Psychology or Mathematics / or our own historiography courses when 
they can be better taught by a specialist in the History depart- 
ment, etc* When we insist on teaching such coiirses when there 
are i5\ibstantial courses in the departments which profess primary 
expertise in those areas, we are wasting our resources and not 
properly- serving our st\idents.^ 
V : There ris another ;^departoenta^^^^^^ 

to be t a^env accpiint pf i^^ any : depar Nqt^] pn ly 

iftust^e^i^assess ^:he 

;§ein/ei the pnee&s^oj^ ahd^ cx:^ll:eges7^ also^ 

:cbnsider the /degree to wh x are apprppriat^^^ f pr 

the student body and t i^t seryes^^^^^^^^ " No longer ;caii 

a department set its goalS : and; determine its /programs: independent 
of community heeds; If a region needs - help -with di^^ the 
developiaent of ski 11. s in organizing vs^^ maintaining groTaps f pT V 
-:cpmmunity action prpgr we of f er only pral intei^retation/ 

/formal piibiievspea^ jthj^ry/^/ j 

/serious questions shpuld be raisjed abput whet^ w*3 deserve the^ v 
/f inanciat support joi^ thaft cpnunujii ty • v ^ /; 



V- : ACCOUNTABILITY " 
. Let us consider now some of the questions raised by the cur- 
rent push for accountability.: "The principle of accountability is 
igbod/ one yith whi of - us can argue/ Clearly / w^^^^ in the 
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colleges and universities must be accountable to the public which 
supports lis and to those agencies which sitand between us eind the 
public. The principle of accoxmtability is fine. I am less sure 
about the practice. In principle/ an accountability/ system measures 
the input-to an institution (the dollars allocated) against the 
output (the amount of education) . In practice ^ our measure of 
dollars seems to be pretty good , but I have serious reservations, 
about the measure of education that we have been using. That mea- 
sure, as far as I can see/ appears to be student credit hpiirs 
generated. The amount o)r quality of learning which each of those 
credit ho;ars__ represents^ seldom i:^_ ever _^ets taken into account . 
Until the^^^^ 

^ng^ ^:he,^wo2:k: ;pf^ coQieges^ a^ -r- 
:^s^lessr/: jif, ncrt:Xseri^ ^partoen^ jor; ar^ 

^ioniiis ; no*/ J:iJc^ia.jdepartment^ : 
is. r^pid/turnov:^r/^riS|L 

Basing^^eyaluations of deparianaents or solely , or even 

pr ima:rily^ on the average cost pf ^student credit^;hour s generated 

^ncourcuges a Vcuciedby ^^prcicrti l^itig ^xraa^ 

L^Lpwer thi^r gucility^ jDf "e^u^ ^(^c>}xca 
to Jceep promotions and/ tenur a minimiM^^ 

of faculty instead because .senior :f^ more money and don't 

generate any more credit ^^^1^^ 

assistant professors. It will l discourage a departaient from- offer- 
ing independent study and small groupys^^ ample 
office hours and -carrying^ on xeseardi^ ^ 
Account^ility procedures >wi 11 not encourage these counter- 
productive practices i^f" we build in^^p^ measures of educational ^ 
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effectiveness- — if we can successfully combine valid measures of 
effectiveness with our measures of efficiency* The problem is 
ficirr^gr that up until now, no institution that I can discover has 
been successful in combining them. 

The reason for this failure I believe is clear. We have been 
trying to measure the effectiveness of education at too macroscopic 
a level; we have been trying to find a single measure which can 
be applied to any of the graduates of an institution. Up; until 
now—and ll see no possibility for predicting anything different 
in the future — colleges ^d universities have been spectaculeirly 
unsuccessful in def iningi t^ ins ti tut ioned goal in avineaning;r 
ful;^wa^-:^in^a way 

tor which those g^^^ Iiaye^beei^^^ reati^st ic/ s^^ 

caai pro^abO^ ney§r TpjpfeA^i 

;siohai_ institutibn^^^ 

spine of the goals ai^^^ clearlyv def 

Otherwise^ especial^ly in. the vl^ cptiegey the educatibhal 

goals^ of the; various, student^^^^ and the yaripus d^ , are so ] 

"heterpgeneous that the :i :have littie-y v 

^ekcept ; to iinpr^s parent ^ ^ 1; 

Eyen:^en such^subjM^^ 
jGraduate ^cordr^ ^he effects of; a pollege V ;- 

.a^e virtually impossibly to^^ Part pf the difficult 

^due to differences in initial: abilities of different cpllege pppu^ 
V lations . However jr eyen when something like the aptitude test of 
:the epllege Entrance Examinitt ion; B^^ the ifimericcm College 

Testing" program is used;:to control for in^ we can 

stil 1 detect . l:ittle effect of : different college^ . The correlations 
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found between these entrance examination tests and the area tests 
is very high — often in the ♦ 9 0 s • - This means that there is not ^ 
much variability left that a college can influence.^ 

Because of these various problems of evaluating effectiveness 
at the moment/ some observers have suggested that while we are 
working on this problem we go ahead and evaluate solely on the 
basis of economic ef f iciency* Off hemd this seems reasonable / 
but there are underlying daaigers. In striving for economic 
efficiency/ we could destroy: some of the major values of higher 
education long before we come up with sat isfactory means of 
:assessing those values so_ that we cem wpr^^ 

y^" :Tn: spite pf t^ 
5^Sr th goveoc^in^r^ ^Modi^ ijSs^txitix^ 
;:compellin5 as we trend :_tow_ardvU^^^ 

^r^s ^or: assessijigvii^ q£-ingty;t\itipn 
:^dp\^edly^ cont inuej/v if ^qt Sc<^3^^ tq aVpid:^^ 

^miniini2e;r.the jd^ the system/: if ^e dqr hotrA/aiit to livey^i^ 

cqs t per student credit hour as the major cril^^^^ of the quality 
of job we are doing / -we must deyelop-valid mea learning 
which can be built in =a& i^^ accountabili^ty . 

f ormuiae. iJot qnly^ would such^^ i^^^ the major cr it ij- 

cisms_ of accountability procedures/ they; would be a positive ^gain^ 
/for they would give students/ faculty/ ad^^^ and thel 

public a reasonable basis for assessing a d^ or institu- 

^tion. ; ' _ ' /~ ; ; ----- ; - ' - ';r^- 

In my opinion/ we will not be able tq develop valid measures: 
of institutionai effect if we cqntinue to cqncentrate qn the very 
.general measur^^s of the impact .of. cqlleges: which have been attempted 
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in the, past* It is probably even useless— at least at this point 
in time — to attempt to develop criteria for departments or pro- 
grams* The futility of attempting this in our field/ for example^ 
is demonstrated by the failure of experts in the Educational Test- 
ing Service/ even with the cooperation of subject-matter experts 
from our field/ to develop a satisfactory area test for speech 

.communication as part of the Graduate Record Examination. There- 
fore/ instead of trying and failing at that task again/ we ought 
to begin with the courses or clusters of courses which tend to be 
taught in most departments or. institutions and develop tests for 

^theitiv^v^l^ catn- b% vus^ :zi^^^uality:^X5f -te^ 

vT measures: ±hat^I>>ai^ 

colriege or. i^^^ abcoiantants or 

Ph.D. ' s in -educatipneil .^to pjemning 

most Qf the: accotmtad^^ shpulxa be-^inter- 
ested lin dping^^ Only those" jof^^^^ 



V£U?ipus subject;^^^ And/^if a:^ 

it meanin^Euliyy: it m This means 

th:at \^;:here fpr^ iStie/ fieldf^of 

speech commiinicatlon / - and^ -that -we can only do v^the - j pb well if_ ^e 
coordinate bur efforts through an organisation such a the Speech 
Communication Associcition or- the -Association for pepartments- and 
Acipainistrators of: Speech Com 

Standardized tests of thi^ sort should be developed^ for all 
of the key courses, or clusters of courses where there is consensus 
among institutions on course gpais:* _ Where, such agreement dpes; 
not exist, we must towarid y bringing . it abput • . : _ - - - 
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Lest I seem' overly optimistic/ I would point out that I re- 
cognize the difficulties of getting agreement on course goals from 
different institutions—even in a field such as ours where we^ 
profess expertise in problem-solving discussion. Certainly our 
past efforts "at assessing the effects of speech communication 
courses have not been very successful • Consider, for example/: 
the research that we have dohe on courses in public speaking. We 
have studied their effect on personality r on critical thinking/ 
on attitudes toward speaking/ on anxiety while speaking , and--once 
in a while — even on skill at speaking. The picture that :CTierg;es 
:^rom research: o^ 

/5 V 3?qL Sevki^p^ 
:?^eiyes bn::v^ 
inust aQjsp trahsrla*^ 

J^dvhere : wej value; vf::?i^^op^ 

;set :of :behavioralv ob jec^^ specificity 
which inakes-^^ iut-this-yery^ a ds^g^^- 

tVhen we insist upon: highly s 
iikely forcing oursel^ 

COTies • Tpx exearvp^l^ ;r\;sophis;ii cated^in educatipna 

evaluation / ; has _-0ven made^^^^^^ comnient a^ of behayipral 

objectives in the-mathematics^^p 

The- unfortunate cohseq^^ that ^" 

the interrelatedness of math concepts is lost and the 
stateanent is a^ tedidus^^^ l^ 
-V : skillsvS;^ : : - ; ^ rr^- - : r: /; T^^^ :^ ^ 

If this is a problem with the behavioral goals for primary school 

mathematics^ where^ t^here- seem to . agreed^ upon^ b^^^ 
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pupils should learn/ how much more of a problem it is bound to 
be at the college level , especially, in a'^field^ such as_ ours* 

In spite of these problems, with the push -from an organiza- 
tion such as ours/ I am confident that we can develop useful 
measures which, for comparative purposes, have a high degree of 
validity* Another of the functions of SGA or ADASC can be to : 
help us gather data from a good sample of departments with similar 
missions in similar institutions. Using the data from each 
cluster of similar institutions/ 'we can develop norms against 
which the effectiveness of the course or cluster in each depart- 
inent can be^l^ -^^v: r - ^ ^ - ^ ^ z^^C^^^ 

With thepe: measure s : of z _ indiyidual courses and clu stars x>f s,::^:r 
courses , we will uitiinately^^^b^^ able: to. get back_ to the_ evaluation ^ 
of total departments/ and even of c^^ uniyersiti^es._^^^A^^^ 
that point though^, we wa^ evaluate these 1^ 

structtires not by using a coinmon jmeasiare for all students within^^ 
them./ but rather by .combi^^ results of -tSojse in^^ 

measures which are most appropriate for each stu]dent iiv each in- 
Jstit\itix>n.^ ^ / ^V:-:^^^ <^-}^/,: 

- After, we . h^ developed ^eliabi^^ yalid measuire^^ eff ec- 
tiveness of courses/ clusters^ of cpurses/ or even :MdepartmentS / ; 
col^ieges/ and universitiesv we will be almost hx>me--thpugh not 
quite in the door . We will s till h^ve the problem of bringing ^ ^ 
together our measures of_ef fectiyeness with t^ effic- 
iency which are now being refined in the acoountcdDility studies^^^^ 
Ultimately/ we will come to the point where we must place a dollar 
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ultimately ^7e must ask of _ any particular 1^^ or any particular i . 
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amount of gain, "is it worth the cost?" For example, how much 
money is it worth to increase the knowledge of 100 students about 
communication processes by 10 per cent? Or how much is it worth • 
to decrease the speech anxiety of 20 students by 5 per cent? When 
we reach these kinds of questiohs, research and r-^e: -hers can 
offer little help._ These questions of relative . .a ate ones 
that must be answered by the society. Hopefully, though, it will 
be an enlightened society. Here also there' is a function for or- 
ganizations such as the SGA and ADASC. These organizations must 
be concerned with helping their members to educate the "general 

-public ^ the decision-makers in ins ti^utions^c^ higher learning,^ 

about Uie^valuer W inCim 

tication o^^ i^^iyiaua^ i ::^:^- 

^ When we have achieved these ends that I have suggested, 
evaluation of our programs and our departments should be a great 
to us and to our field. , v > v • ^ 
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6. The problem of standardizing these varied measures so that th'ey 
are comparable will be relatively simple since we will be con- 
concerned with deviations from norms rather than absolute scores. 



